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The  problem  of  estimating  the  mean  of  a p-variate  normal 
distribution  has  been  of  considerable  interest  to  the  statis- 
ticians since  the  pioneering  work  of  Stein  who  showed  that 
the  maximum  likelihood  estimator  (MLE)  is  inadmissible  with 
respect  to  a quadratic  loss  function  when  p ^ 3.  Certain 
families  of  estimators  have  been  shown  in  the  literature  to 
dominate  the  MLE.  In  this  paper  we  consider  the  case  in 
which  the  covariance  matrix  of  the  normal  distribution  is 
singular.  An  application  of  the  given  result  arises  in  a 
problem  of  estimating  the  mean  of  a multinomial  distribution. 
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1.  Introduction.  Let  X be  a K-component  (K  >_  3)  random  vector 
distributed  according  to  a multivariate  normal  distribution 
N (y  , Z ) with  mean  y and  covariance  Z . For  estimating  u lot  the 
loss  function  be  given  by 

, (1.1)  L(6;y,Z)  - (5-y)  'A(S-y) 

where  6 ■ 6 (X)  denotes  *ny  estimator  of  y,  and  A is  a given 

symmetric  positive  (semi-positive)  definite  matrix.  The  maximum 

likelihood  estimator  (MLE)  is  the  vector  X and  is  known  to  be  mini- 

2 

max.  On  the  other  hand,  for  A * I and  Z * a I,  where  I denotes  the 
identity  matrix,  Stein  (1955)  showed  that  the  MLE  is  inadmissible 
with  respect  to  the  given  loss  function.  Since  the  pioneering 
work  of  Stein,  the  given  problem  has  been  examined  by  various 
authors.  They  have  considered  certain  families  of  estimators 
which  are  shown  to  dominate  the  MLE.  They  are  therefore  minimax. 
The  papers  of  Alam  (1977)  and  Efron  and  Morris  (1973,  1976)  may 
be  cited  for  reference.  A list  of  other  papers  may  be  seen  in 
the  bibliography  given  in  the  two  papers. 

In  the  papers  cited  above,  the  minimax  estimators  are  given 
for  the  case  in  which  the  covariance  I is  a non-singular  matrix. 

In  this  paper  we  consider  the  case  in  which  Z is  singular.  Two 
cases  may  be  considered:  (i)  Z is  known  and  (ii)  Z is  unknown, 
but  an  estimate  S/m  is  given,  where  S is  distributed  independent 
of  X according  to  a Wishart  distribution  W(S:Z,m).  Let 
0:  (0,»)  -►  (0,1).  If  Z is  non-singular,  consider  a class  of 
estimators , given  by 


(2) 


1 

« 

-1 

(1*2)  6 (X)  - $(X,I  xX)X  for  case  (i)  , and 

(1.3)  n(X)  - 4»(X'S_1X)X  for  case  (ii)  . 

The  author  has  shown  (Alam  (1977),  Theorems  2.1  and  2.2*)  that 
<S  and  n dominate  the  MLE  for  a certain  class  of  functions  $ . 

In  the  following  section  we  shall  extend  the  given  result  to 
the  case  in  which  E is  singular.  For  this  case,  the  estimators 
are  given  by  substituting  E~  for  E-1  in  (1.2)  and  s"  for  s”1 
in  (1.3),  where  E and  S~  are  generalized  inverse  of  E and  S, 
respectively,  satisfying  the  relations  EE~E  » E and  SS~S  * S. 

An  application  of  the  given  result  arises  in  the  problem 
of  estimating  the  mean  of  a multinomial  distribution  M(n,p) , 

I 

where  p « (p^,...,p^)  denotes  the  cell  probabilites  and  n 
represents  the  total  of  cell  frequencies.  To  see  this,  let 
X ? M(n,p)  where  - means  "distributed  as".  The  vector  X/n 
is  the  maximum  likelihood  estimator  of  p,  its  covariance  is  a 
singular  matrix  and  it  is  asymptotically  normally  distributed 
for  large  n. 

2.  Main  results.  First  let  E be  known.  Consider  the  esti- 
mator 5 , given  by  (1.2),  with  the  substitution  of  E~  for  E_1, 
where  E~  is  a generalized  inverse  of  E.  Let  X ■ QY,  where  » 

Y ~ N (v,I) , QQ'  - E and  u - Qv.  Since  EE*E  - E or  QQ ' E ~QQ ' - 
QQ' , we  have  Q'E-QQ'  ■ Q'.  Therefore 

Q'E"QQ'E"q  - Q'E“Q. 


(2.1) 


(3) 


That  is,  Q'E  Q is  an  idempotent  matrix.  Also 

(2.2)  Rank  Q'E'q  * Rank  E * l,  say. 

Since  EAEE~E  - EE~EAE  , or 

Q(Q'AEE‘q  - Q'E^AQJQ'  - 0 

we  have 

(2.3)  Q ' AQQ ' E~Q  - Q'E’QQ'AQ. 

That  is,  the  matrices  Q'AQ  and  Q'E  Q commute.  Therefore,  there 
exists  an  orthogonal  matrix  P,  say,  which  diagonalizes  simul- 
taneously the  two  matrices.  Since  Q'E~Q  is  idempotent,  we  have 

P'QE_QP  - 1^ 

P'Q'AQP  - D 

where  1^  denotes  a K*K  diagonal  matrix  with  l elements  on  the 
diagonal,  each  equal  to  1 and  the  remaining  elements  equal  to 
zero,  and  D denotes  a diagonal  matrix  of  rank  r,  where 

r ■ Rank  Q'AQ  _<  Rank  Q * Rank  QQ'  ■ l. 

The  risk,  that  is,  the  expected  loss  of  5 is  giv»n  by 

* • 

(2.4)  R5  - E ( 6 (X)  - M)  'A(6(X)-m) 

- E (<J>(Y’Q'  E_QY)Y-v)  'Q'AQ(<MY'Q'  E*QY)Y-v) 

- E UU'I^jZ-y)  'D(4.(Z'I£Z)2-y) 


(4) 


where  y ■ P'v  and  Z - P'Y  ^ N(Y,I).  Without  loss  of  generality, 
we  can  assume  that  the  first  l diagonal  elements  of  Ij  are  each 
equal  to  1 and  that  di  - 0 for  i > l where  di  denotes  the  ith 
diagonal  element  of  D.  Let  W - (Z^  , . . . ,Z^)  Y*  » (Y1#...»Yt)' 
and  let  D*  be  obtained  from  D by  removing  the  last  K-4  rows  and 
columns.  Then 

(2.5)  R5  - E ($(W’W)W-Y*)'D*(MW'W)W-Y*). 

Let  a^,...,a^  denote  the  characteristic  roots  of  AI  or 
equivalently  of  Q'AQ.  Let  \)>(AE)  « trace  AE  and 
[it  (aE) 

oiq  * y j / maxfc^, . . . ,a^)  £ 1.  Observe  that  the  risk  of  the 

MLE  is  equal  to  i|>(AE)  . Comparing  (2.5)  with  (2.6)  of  [1]  we 
obtain  the  following  generalization  of  Theorem  2.1  of  [1]. 

Theorem  2.1.  Let  l £ 3.  Then  R^  - <MAE)  £ 0 if 
(i)  xt+1(l-<Mx) ) is  nondecreasing  in  x,  and  (ii)  0 £x(l-<t>(x))  £ 
2agJl-4t-4  for  some  value  of  t > 0. 

When  E is  unknown  we  consider  the  estimator  n,  given  by  (1.3) 
with  the  substitution  S~  for  where  S~  denotes  a generalized 
inverse  of  S.  We  obtain  as  above  the  following  generalization 
of  Theorem  2.1*  of  [1]. 

Theorem  2,1*.  Let  l >_  3.  Then  R^-i/)(AE)  £ 0 if 
(i)  xt+1(l-$(x) ) is  nondecreasing  in  x,  and  (ii)  0 £ 

(m-n+2t+3)  x(l-$  (x) ) £ 2ag£-4t-4  for  some  value  of  t _>  0. 


(5) 


3.  Application,  Consider  the  multinomial  distribution  M(n,p) 
with  Jc  cells,  and  let  X ~ M(n,p).  The  covariance  matrix  of  X 
is  given  by  L 3 (0^),  where  °ij  * "npipj  and 

0^  * np^(l-p^).  Clearly,  E is  singular.  For  estimating  np 
let  the  loss  be  given  by  (1.1).  Two  cases  are  considered: 

(a)  A » n *1  and  (b)  A = n~^C,  where  C is  a diagonal  matrix 
whose  ith  diagonal  element  is  equal  to  p^-1.  In  case  (a)  the 
loss  is  proportional  to  the  sum  of  squared  errors.  Since  n~^C 
is  a generalized  inverse  of  E,  the  loss  in  case  (b)  is  pro- 
portional to  Mahalanobis  distance  function.  The  loss  due 
to  the  MLE  in  case  (b)  leads  to  Pearson's  Chi-square  statistic, 
used  for  the  goodness  of  fit  test.  Since  X is  asymptotically 
normally  distributed  for  large  values  of  n,  Theorem  2.1  might 
be  used  to  improve  upon  the  maximum  likelihood  estimator  X. 

For  the  application  of  the  theorem  it  should  be  noted  that 
L * rank  E * k-1  and  that  the  characteristic  equation  of 
n”^E  is  given  by 

2 

(3-1>  11  - Ji-i  p^r»  "5-i  - »• 

Let  denote  the  largest  characteristic  root  of  n**1!.  From 
(3.1)  we  have  that  — p[k]'  where  p[i]  denotes  the 

ith  smallest  value  amongst  p^,...,p^.  Therefore,  in  the  case  (a) 

°0  * 


a - Ii.x  PiVvk-l). 


(6) 


In  the  case  (b) , the  largest  characteristic  root  of  AI  is  equal 
to  1,  and  aQ  * 1. 

k — 3 

Consider  the  case  (b)  . Let  <(>  (x)  = 1 - Then 

x 

k-3  I 

(3.2)  6 (x)  = (1  - 

Since  the  diagonal  matrix  with  the.ith  diagonal  element  equal  to 
(np^)  1 is  a generalized  inverse  of  E,  we  substitute  T-1  for  Z~  in 

(3.2)  where  T denotes  a diagonal  matrix  whose  ith  diagonal  element 
is  equal  to  n^,  the  maximum  likelihood  estimate  of  np^.  Then 

the  risk  of 

(3.3)  5 (x)  = (1  - ^h)X 


= (1  - — ) X 

with  respect  to  the  loss  (1.1)  with  A = n_1C  is  given  by 

(3.4)  R - oc-l,  - + 

5 n n2 

< (k-1)  = Rx 

for  n > (k-1) (k+3) /<k+l) . Therefore,  the  MLE  is  inadmissible, 
being  dominated  by  6.  The  estimate  can  be  further  improved  by 
letting 

6 (x)  = (1  - V(k“3)  )X 
X'z  X 

and  minimizing  for  v.  The  minimizing  value  of  v is  given  by 


[ 


(7) 


k-3  k-1. 

iTT  (1  + — } 


-1 


k-2 


Remark:  The  choice  of  4>  ( x ) » 1 - in  (1.2)  for  estimating 

the  mean  of  a multivariate  normal  distribution  with  non-singular 
covariance  matrix  was  originally  proposed  by  James  & Stein  (1961) . 
It  should  be  noted,  however,  that  the  relation  (3.4)  establishing 
the  inadmissibility  of  the  MLE  is  not  based  on  the  asymptotic 
property  of  the  multinomial  distribution. 


1 


V 


(8) 
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